近似消息传递（AMP）类型算法已被广泛用于某些大型随机线性系统的信号重建。AMP型算法的关键特征是可以通过状态进化正确描述其动力学。但是，状态进化不一定保证迭代算法的收敛性。为了解决原则上AMP类型算法的收敛问题，本文提出了在足够的统计条件下的记忆AMP（MAMP），称为足够的统计MAMP（SS-MAMP）。我们表明，SS-MAMP的协方差矩阵是L带和收敛的。给定任意启动，我们可以通过阻尼来构建SS-MAMP，这不仅可以确保收敛性，而且可以保留正交性，即可以通过状态进化正确描述其动力学。

近似消息传递（AMP）是一种希望具有非高斯信令的某些高维线性系统的未知信号重建的有希望的技术。 AMP型算法的杰出特征是它们的动态可以通过状态演进来严格描述。但是，状态的进化不一定保证迭代算法的融合。为了解决AMP型算法的收敛问题原则上，本文提出了一种在足够的统计条件下的存储放大器（MAMP），命名为足够的统计MAMP（SS-MAMP）。我们表明SS-MAMP的协方差矩阵是L-带状和会聚。考虑到任意的MAMP，我们可以通过阻尼构造SS-MAMP，这不仅可以确保MAMP的收敛，而且还可以保留MAMP的正交性，即，其动态可以通过状态演变严格地描述。作为副产品，我们证明贝叶斯最佳正交/载体放大器（Bo-Oamp / Vamp）是SS-MAMP。结果，我们揭示了大型系统的Bo-Oamp /鞋面的两个有趣特性：1）协方差矩阵是L型带状的，并且在BO-Oamp / vamp中收敛，2）阻尼和存储器无用（即，做在BO-OAMP / VAMP中没有带来性能改进。作为一个例子，我们构建了一个足够的统计贝叶斯 - 最佳MAMP（BO-MAMP），如果其状态进化具有独特的固定点，并且其MSE比原来的BO-MAMP更糟糕，那么它是最佳的。最后，提供了模拟以验证理论结果的有效性和准确性。

近似消息传递（AMP）是具有非高斯分布的某些高维线性系统的低成本迭代参数估计技术。然而，放大器仅适用于独立相同的分布（IID）变换矩阵，但是对于其他矩阵集合，尤其是对于不良条件的矩阵，可能变得不可靠（例如，表现不良或甚至不同）。建议正交/矢量放大器（OAMP / VAMP）用于一般右单一不变的矩阵来处理这种困难。然而，贝叶斯最优休息/鞋面（BO-OAMP / VAMP）需要高度复杂性线性最小均方误差（MMSE）估计器。这限制了oamp / vamp在大规模系统中的应用。为了解决AMP和BO-OAMP / VAMP的缺点，本文提出了在正交原理下的记忆放大器（MAMP）框架，保证了MAMP中估计估计误差的渐近IID高斯。我们为本域内存估算器提供了一个正交化过程，以实现MAMP所需的正交性。此外，我们提出了一种贝叶斯 - 最佳机制（BO-MAMP），其中提出了一种用于干扰抑制的长存储器匹配过滤器。 BO-MAMP的复杂性与AMP相当。源于渐近表征Bo-MAMP的性能的状态演变。基于国家演化，优化了BO-MAMP中的松弛参数和阻尼载体。对于所有右单一不变的矩阵，优化的BO-MAMP的状态演变会收敛到与高复杂性BO-OAMP / VAMP相同的固定点，并且如果其状态进化具有独特的固定点，则是贝叶斯的最佳状态。最后，提供了模拟以验证理论结果的有效性和准确性。

This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.

Supervised Question Answering systems (QA systems) rely on domain-specific human-labeled data for training. Unsupervised QA systems generate their own question-answer training pairs, typically using secondary knowledge sources to achieve this outcome. Our approach (called PIE-QG) uses Open Information Extraction (OpenIE) to generate synthetic training questions from paraphrased passages and uses the question-answer pairs as training data for a language model for a state-of-the-art QA system based on BERT. Triples in the form of <subject, predicate, object> are extracted from each passage, and questions are formed with subjects (or objects) and predicates while objects (or subjects) are considered as answers. Experimenting on five extractive QA datasets demonstrates that our technique achieves on-par performance with existing state-of-the-art QA systems with the benefit of being trained on an order of magnitude fewer documents and without any recourse to external reference data sources.

Transformer has achieved impressive successes for various computer vision tasks. However, most of existing studies require to pretrain the Transformer backbone on a large-scale labeled dataset (e.g., ImageNet) for achieving satisfactory performance, which is usually unavailable for medical images. Additionally, due to the gap between medical and natural images, the improvement generated by the ImageNet pretrained weights significantly degrades while transferring the weights to medical image processing tasks. In this paper, we propose Bootstrap Own Latent of Transformer (BOLT), a self-supervised learning approach specifically for medical image classification with the Transformer backbone. Our BOLT consists of two networks, namely online and target branches, for self-supervised representation learning. Concretely, the online network is trained to predict the target network representation of the same patch embedding tokens with a different perturbation. To maximally excavate the impact of Transformer from limited medical data, we propose an auxiliary difficulty ranking task. The Transformer is enforced to identify which branch (i.e., online/target) is processing the more difficult perturbed tokens. Overall, the Transformer endeavours itself to distill the transformation-invariant features from the perturbed tokens to simultaneously achieve difficulty measurement and maintain the consistency of self-supervised representations. The proposed BOLT is evaluated on three medical image processing tasks, i.e., skin lesion classification, knee fatigue fracture grading and diabetic retinopathy grading. The experimental results validate the superiority of our BOLT for medical image classification, compared to ImageNet pretrained weights and state-of-the-art self-supervised learning approaches.

Knowledge graph embedding (KGE), which maps entities and relations in a knowledge graph into continuous vector spaces, has achieved great success in predicting missing links in knowledge graphs. However, knowledge graphs often contain incomplete triples that are difficult to inductively infer by KGEs. To address this challenge, we resort to analogical inference and propose a novel and general self-supervised framework AnKGE to enhance KGE models with analogical inference capability. We propose an analogical object retriever that retrieves appropriate analogical objects from entity-level, relation-level, and triple-level. And in AnKGE, we train an analogy function for each level of analogical inference with the original element embedding from a well-trained KGE model as input, which outputs the analogical object embedding. In order to combine inductive inference capability from the original KGE model and analogical inference capability enhanced by AnKGE, we interpolate the analogy score with the base model score and introduce the adaptive weights in the score function for prediction. Through extensive experiments on FB15k-237 and WN18RR datasets, we show that AnKGE achieves competitive results on link prediction task and well performs analogical inference.

Digital engineering transformation is a crucial process for the engineering paradigm shifts in the fourth industrial revolution (4IR), and artificial intelligence (AI) is a critical enabling technology in digital engineering transformation. This article discusses the following research questions: What are the fundamental changes in the 4IR? More specifically, what are the fundamental changes in engineering? What is digital engineering? What are the main uncertainties there? What is trustworthy AI? Why is it important today? What are emerging engineering paradigm shifts in the 4IR? What is the relationship between the data-intensive paradigm and digital engineering transformation? What should we do for digitalization? From investigating the pattern of industrial revolutions, this article argues that ubiquitous machine intelligence (uMI) is the defining power brought by the 4IR. Digitalization is a condition to leverage ubiquitous machine intelligence. Digital engineering transformation towards Industry 4.0 has three essential building blocks: digitalization of engineering, leveraging ubiquitous machine intelligence, and building digital trust and security. The engineering design community at large is facing an excellent opportunity to bring the new capabilities of ubiquitous machine intelligence and trustworthy AI principles, as well as digital trust, together in various engineering systems design to ensure the trustworthiness of systems in Industry 4.0.

Surgical robot automation has attracted increasing research interest over the past decade, expecting its huge potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied AI has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant researchers. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how human demonstrations would affect policy learning. In this paper, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. With these, we further propose to collect human demonstrations and imitate the action patterns to achieve more effective policy learning. We showcase the improvement of our simulation environment with the designed new features and tasks, and validate state-of-the-art reinforcement learning algorithms using the interactive environment. Promising results are obtained, with which we hope to pave the way for future research on surgical embodied intelligence. Our platform is released and will be continuously updated in the website: https://med-air.github.io/SurRoL/

Learning the underlying distribution of molecular graphs and generating high-fidelity samples is a fundamental research problem in drug discovery and material science. However, accurately modeling distribution and rapidly generating novel molecular graphs remain crucial and challenging goals. To accomplish these goals, we propose a novel Conditional Diffusion model based on discrete Graph Structures (CDGS) for molecular graph generation. Specifically, we construct a forward graph diffusion process on both graph structures and inherent features through stochastic differential equations (SDE) and derive discrete graph structures as the condition for reverse generative processes. We present a specialized hybrid graph noise prediction model that extracts the global context and the local node-edge dependency from intermediate graph states. We further utilize ordinary differential equation (ODE) solvers for efficient graph sampling, based on the semi-linear structure of the probability flow ODE. Experiments on diverse datasets validate the effectiveness of our framework. Particularly, the proposed method still generates high-quality molecular graphs in a limited number of steps.